Knowledge-Constrained Answer Generation for Open-Ended Video Question Answering

نویسندگان

چکیده

Open-ended Video question answering (open-ended VideoQA) aims to understand video content and semantics generate the correct answers. Most of best performing models define problem as a discriminative task multi-label classification. In real-world scenarios, however, it is difficult candidate set that includes all possible this paper, we propose Knowledge-constrained Generative VideoQA Algorithm (KcGA) with an encoder-decoder pipeline, which enables out-of-domain answer generation through adaptive external knowledge module multi-stream information control mechanism. We use ClipBERT extract video-question features, framewise object-level from commonsense base compute contextual-aware episode memory units via attention based GRU form exploit mechanism fuse features such semantic complementation alignment are well achieved. evaluate our model on two open-ended benchmark datasets demonstrate can effectively robustly high-quality answers without restrictions training data.

برای دانلود رایگان متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Open-Ended Visual Question-Answering

This thesis studies methods to solve Visual Question-Answering (VQA) tasks with a Deep Learning framework. As a preliminary step, we explore Long Short-Term Memory (LSTM) networks used in Natural Language Processing (NLP) to tackle Question-Answering (text based). We then modify the previous model to accept an image as an input in addition to the question. For this purpose, we explore the VGG-1...

متن کامل

Proposing Plausible Answers for Open-ended Visual Question Answering

Answering open-ended questions is an essential capability for any intelligent agent. One of the most interesting recent open-ended question answering challenges is Visual Question Answering (VQA) which attempts to evaluate a system’s visual understanding through its answers to natural language questions about images. There exist many approaches to VQA, the majority of which do not exhibit deepe...

متن کامل

Answer Formulation for Question-Answering

In this paper, we describe our experimentations in answer formulation for question-answering (QA) systems. In the context of QA, answer formulation can serve two purposes: improving answer extraction or improving human-computer interaction (HCI). Each purpose has different precision/recall requirements. We present our experiments for both purposes and argue that formulations of better linguisti...

متن کامل

Question Generation for Question Answering

This paper presents how to generate questions from given passages using neural networks, where large scale QA pairs are automatically crawled and processed from Community-QA website, and used as training data. The contribution of the paper is 2-fold: First, two types of question generation approaches are proposed, one is a retrieval-based method using convolution neural network (CNN), the other...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

ژورنال

عنوان ژورنال: Proceedings of the ... AAAI Conference on Artificial Intelligence

سال: 2023

ISSN: ['2159-5399', '2374-3468']

DOI: https://doi.org/10.1609/aaai.v37i7.25983